DOCUMENT RESUME 



ED 034 686 



24 



SE 007 688 



AUTHOR 

«r>T T>T T? 



INSTITUTION 

S p ONS AGENCY 

BUREAU NO 
PUB DA"? 

NO^E 



Gordon, C. Wayne 

Preliminary Evaluation Report on the Los Angeles 
City Schools, SB 23 Demonstration Program in 
Mathematics. 

California T Jniv. , Los Angeles. Center for the Study 
of ^valuation. 

Office of Education (DHEW) , Washington, D.C. Bureau 
of Research. 

RR-6-1646 
Oct 69 
66d. 



EDRS PRICE EDRS Price MF-SQ.50 HC-S3.40 

DESC PT PTOFS *Educational Research, *Exper imental Programs, Grade 

7, Grade 8, Instruction, Instructional Materials, 
Material Development, *Matheraatics, *Program 
Evaluation 



ABSTRACT 



The purpose of this preliminary report is to 
describe and evaluate the los Angeles Model Mathematics Project 
(LAMMP) . The objectives o* this project include the improvement of 
mathematical skills and understanding of mathematical concepts, the 
improvement of students* self-image, the development of instructional 
materials and the assessment of commercially developed instructional 
materials. The report describes the manner by which participants in 
the project were chosen and the instructional programs which were 
employed. An evaluation of measurement instruments used, a summary 
and interpretation of evaluation data, and recommendations for 
improving the project are also included. Only a * preliminary analysis 
of data is contained in this report and no attempt is made to make 
statistical inferences. T he report concludes that in many cases 
subjects were not randomly assigned to experimental and comparison 
groups and thus no valid inferences about the effects of 
instructional variables based on differences between experimental and 
comparison grouos can be made. ( F L) 
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I. Objectives 

The objectives of the Los Angeles Model Mathematics Project, 
hereafter referred to as LAMMP, were stated by the admin is ti ati on 
of the project (L. A. City Schools publication, April 12, 1967) 
OBJECTIVES 



To improve mathematical skills and understandings of mathe- 
matical concepts 

To improve the pupils' self-image 

To identify specific assets and limitations relating to the 
learning process 

To develop and use special instructional materials and pro- 
grams and to assess their Values 

To select and use appropriate commercially developed equip- 
ment, instructional materials, and programs and to assess 
their values. 
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A. 



Ch ore • cte r isti c s 

1. The participants were 7th and 8th grade students. 

The distribution of students by grade level is shown 
in Table 1. 



Table 1 

Grade Levels of Participants 







Fall, 


1967 




Spring 


, 1968 




Belvedere 


B7 


A 7 


Total 


B7 


A7 


21 


Total 


No. of classes 


6 


2 


8 


0 


6 


2 


8 


No. of students 


115 


37 


152 


0 


111 


37 


148 


Edison 
















No. of classes 


6 


0 


6 


2 


4 


0 


6 


No. of students 


88 


0 


88 


40 


75 


0 


115 


Pacoima 
















No. of classes 


6 


2 


8 


0 


6 


2 


8 


No. of students 


116 


38 


154 


0 


106 


35 


141 




No 


. of students, all 


schools 








B 1 _ 


A 7 








21 


Total 


Fall, 1967 


319 


75 










394 


Spring, 1968 


40 


292 








72 


404 



2 . 



School 

Bc.i.vedere 

Edison 

Pacoima 



School 

Belvedere 

Edison 

Pacoima 



The distribution of participants by 


sex is 


shown in 


Table 2. 
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Sex of Participants 






Fall 


, 1967 


Spring 


, 1968 


% Girls 


% Boys % 


Girls 


% Boys 


53 


47 


52 


48 


52 


48 


47 


53 


46 


54 


45 


55 



The distribution of participants by ethnic groups is 
shown in Table 

Table 3 

Ethnic Groups Among Participants 



Caucasian 


% Negro 


% Mexican -American 


% Other 


2 


2 


95 


1 


0 


95 


5 


0 


58 


2 


39 


1 
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4. An estimate of the ability levels of participants 
was available from intelligence quotients computed 
from a fifth grade administration of the California 
Test of Mental Maturity. Information from this test 
is summarized in Table 4. 

Tabic 4 



Mean I.Q. by School for Participants 
on the California Test of Mental Maturity 
(1957 Short Form) 



School 


Verbal 

Mean 


Non-Verbal 

Mean 


Total 

Mean 


Belvedere 


93.6 


94.1 


94.0 


Edison 


92.6 


86.2 


89.6 


Pacoima 


95.6 


93.2 


94.6 
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g. Procedin es uscd to select stu den ts. The target group for the 
program was students of average ability who were achieving in math- 
ematics a year or more be lot: grade level. The procedures used to 
select students for the program were determined by the counselor 
consultant for the project, and the head counselor in each of the 
three schools. The criteria used to identify eligible students 



v:ere described by the counselor consultant as follows: 

1. Used tost information CTMM and CAT scores from pupils* 

1905 A5 evaluations. 

2. An effort was made to consider current mid-term mathematic 
grades . 

3. Consideration was given to the fact that standardized 
group tests do not always reflect these children* s full 
potential . 

4. When current group IQ tests did not indicate average 
ability, scores from individual tests or earlier non- 
verbal tests were used. 

5. Average intelligence was interpreted to be that which 
represents average intelligence for that school. 

6. When unusual score patterns were found, counselor and 
teacher consultation was sought. 

The procedures followed in selecting students for the program 

were reported as follows by the project counselor. 

Edison Junior High School The B7 students identified as 
eligible were scheduled into classes of 40. The first day 
of school an experimental and a comparison teacher "randomly" 
divided each of these classes. The rate of attrition at this 
school resulted in small experimental and large comparison 
classes by the end of the first semester; new students were 
scheduled into the comparison classes. 

At the beginning of the second semester, the original six 
experimental classes were "randomly" combined into four 
classes and two new B7 classes of 40 were formed and divided 
in the same maimer as described above. 

Paco j mo Junior H igh School The students identified as eli- 
gible were scheduled into six B7 and two A7 experimental 
classes. Four comparison classes (three B7 and one A7) were 
formed from among other students considered to have "average" 
academic capability. 
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The saisie experimental students were* "randomly" reassigned to 
experimental classes the second semester. A number of com- 
parison students were assigned to "average" level math classes 
not designated as comparison classes and their place filled 
by other students considered to have average academic capa- 
bility. 



Belve d ere Junior High Scho ol A6 students identified as 
eligible were randomly scheduled into experimental and com- 
parison B7 classes by the head counselor. The A7 classes 
were formed by the counselor consultant, who used a table 
of random numbers to assign eligible students to experimental 
and comparison groups. 

Aside from the few students who transferred during the year, 
experimental and comparison classes were maintained intact 
throughout the year. 

C. Diagnosis of mathe ma tics problems . Any diagnosis of students' 
difficulties in learning mathematics were carried out by each 
teacher for his own students. No systematic diagnosis procedures 
were followed by all p7'oject personnel. 

Students' total scores from the Iowa Tests of Arithmetic and 



Vocabulary and the LAMMP Diagnostic Test, which were administered 
by the evaluation team in September, were made available to the 
teachers. Teachers used the test information as they wished in 
diagnosis of students' mathematics skills. 



Although the project plans called for a counselor consultant 
to assist teachers in the diagnosis of individual students' needs, 
this assistance was not in fact available to the teachers until 



February 1968. The first counselor employed left the project 
before school opened in September 1967, and the position was not 
filled again until the beginning of the second semester. 
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III . Metho ds 

A. In f true t io n a 1 mo t hods . The focus for each of the three 
experimental centers was described as follows. (L. A. City publi- 
cation, April 12, 1967) 

Opera t j onal Objecti ves of the Pro grammed Learning Center at 
Kdi s on ~ J r . H i g ii : 

To select and use comracrcially developed instructional 
materials and programs, and to assess their values 
To develop and use special instructional materials and 
assess their value 

Operational Objectives of t he Cl osed -Ci rcuit Television 
Retrieval Center at Pacoima Jr. 'H igh: 

To measure the changes in student and teacher behavior 
which occur due to availability and use of a retrieval 
system 

To provide teachers and other school personnel an aware- 
ness of improvements of teaching, for introduction of 
new methodology and technology 

To identify specific skills and to train teachers to 
utilize effectively information retrieval services 
To provide a variety of supplementary instructional 
materials to students and teachers 

To produce appropriate lessons on video tape not avail- 
able commercially 

To provide teachers with the skills needed to design 
appropriate individual study programs utilizing all 
instructional resources 

Operational Objective s of the Mathematics Laboratory at 
Belvedere Jr. High : 

To study the effects of a multi -media, multi -sensory 
environment on the mathematical achievement of disadvan- 
taged pupils 

To develop learning materials which are suitable for 
use with pupils in target areas 

In general, the instructional program at each center was 
oriented to the focus described, except at Pacoima. There it was 
not possible to carry out this year a program based on a closed- 
circuit television retrieval system because the necessary equip- 
ment was not installed. The teachers at the Pacoima center 
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developed lessons to be used later in the television system. During 
the second semester some of these lessons were tested with one 
television camera and a videotape recorder. 

Observations of classrooms by the evaluation team indicated 
that experimental classrooms were characterized by: 
flexible classroom organization for instruction; 
use of a wide variety of instructional techniques and mate- 
rials ; 

emphasis on developing students* understanding of basic 
mathematical principles; 

teacher efforts to develop positive concepts in students and 
positive attitudes toward school. 

One objective of the program was to develop new instructional 
materials. Each teacher was originally assigned only two classes 
per day in order to allow time for work on materials. (At mid- 
year, however, the shift of one of the teachers from Pacoima to 
central office staff made it necessary to increase the load of 
two teachers there to 2 > classes per day.) About half of the time 
of the illustrators who were employed by the project was devoted 
to helping teachers prepare materials. 

A mathematics consultant was also assigned to the central 
project staff to work with teachers in all three centers. Obser- 
vations over the school year, however, indicated that, in fact, 
the mathematics consultant spent at least half (often more) of 
his time in administrative duties which did not assist teachers 
in the preparation of instructional materials. 
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Although some materials were erected, .it is impossible to 
evaluate their effectiveness at this time. Samples of the mate- 
rials developed and used during each semester were not submitted 
to the evaluation team until near the end of each semester. 
Therefore, it was not possible to carry out any kind of evaluation 
of the effectiveness of these materials apart from the rest of the 
instructional program. Two units of materials were field tested 
by the project in a few ether schools in the district during spring 
semester, but complete results of these field tests a.rc not yet 
available . 



B. Non- instructional activities . Three major types of non- 
ins true tional activities were carried out in the project: 1) initial 

training workshops for the project teachers; 2) inservice training 
activities for project teachers throughout the year; 3) production 
of public relations materials about the project. 

1. Los Angeles City training workshop . When the project 
teachers were first brought together in February, 1967, they 
participated in a four-week workshop designed specifically as an 
orientation to this project. The workshop program included: 

a) lectures and discussions on teaching mathematics 

b) demonstrations of teaching mathematics 

c) lectures and discussions on working with dis- 
advantaged youth 

d) field trips to resource centers 

e) lectures and practice sessions in developing 
objectives for the program 

f) discusssions with community groups 

g) one-day teaching in experimental schools 

h) sensitivity training 
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lVork sho p o n fount. l ive evsJ n at i on » During the summer 
the evaluation staff din octet! a workshop for the LAMMP teachers, 
in which Dr. Benjamin Bloom was brought in to discuss and demon- 
strate his theory of formative evaluation. 

2. inservi ce training. These activities throughout 

the year included field trips to resource centers, observation and 
discussion of demonstration lessons by visiting mathematics con- 
sultants, and working with consultants on materials which the 
project teachers were developing. About a dozen outside consult- 
ants in the teaching of mathematics were brought in for 1 to 5-day 
periods during the year. About 15-20 district personnel also 
provided input at various times for project teachers. 

Sensitivity training was also continued to some extent after 
the initial workshop experience. During the year the trainers 
worked with each center rather than with the whole project group 
together . 

3. Production o f public relations materials . Approx- 
imately one fifth to one -half the time of the illustrators who 



were employed on the project was spent in preparing explanatory 
brochures and charts about the project. Three brochures describ- 
ing the program at each center were prepared for distribution in 
the community. Other materials described the organization and 
objectives of the total project. Visibility for the project was 
also sought through some television coverage and various kinds 
of publicity within the district. Perhaps the greatest effort 
was made in bringing visitors to the centers. A very large number 
of peop! e- - from within the district and from outside- -visited the 
three centers this year. 
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1 V . Evaluation 

A . Plan f or ev al uat ion 

1. Measurement instruments used 

a) Standardized 

The Iowa Test of Basic Shills was used to collect 
some data for evaluation. Two subtests, the Verba] 
and the Arithmetic (sixth Grade Level) were 
administered in September 1967 to students in 
experimental and comparison classes. The Arithmetic 
Subtest was given again in May, 1968. 

b) Non- Standardized 

Four types of non- standard! zed instruments were 
administered as a part of the evaluation. 

(1) The LAMMP Diag n ost i c T est , a measure of 
achievement in a variety of mathematical skills, 
was constructed from items judged to be pertinent 
to the instructional goals of the LAMMP project. 

The development of this instrument originated in 
recognition on the part of the evaluation staff 
that standardized achievement tests may be generally 
insensitive to relatively short-term learning 
experiences because their items are mainly irrelevant 
to the particular instructional program under study. 
Selection of the items was based on a two- 
dimensional classification system, shown in Figure 1, which 
organizes test items into classes of operational objectives 
defined by the intersection of content and process dimensions. 
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For example, one such intersection is "integers" (content) 
and 'l.lussif ication" (process). An item from this cell is, 

Which numeral has been left out? 

5 10 20 25 30 

a. 11 

b. 15 

c. 19 

d. 100 

It was hope.d that reference to this organization 
of test content would result in an instrument useful for 
comparing groups of students with respect to specific areas 
of achievement in mathematics. The broad coverage of the 
LAMMP Diagnosti c Test in terms of types of mathematical 
skills contrasts with the typical standardized ahievement 
test. To maximize the accuracy of statements about the 
performance of individual students, the iatter often con- 
tain many items of the same type. 

Two forms (Form A and Form B) of the LAMMP Diagnostic 
Test were constructed by randomly assigning members of item 
pairs selected from the same content-process category. The 
49 starred cells of the content-process matrix presented 
in Figure 1 are represented by items on the LAMMP D iagn o stic 
Test. The two forms of the test were constructed so as to 
reduce the amount of testing time and still provide for 
replication of particular item types. Forms A and B were 
randomly assigned to classes within experimental and 




comparison groups . 



] tern? of the b AMMP _ iHf L n ( M: t Ail .Ifbll ' :eie organized 
i n 1. o t In e c s c c t i o n z . The first, c on t a i n :i n g i t cm s 3-40, 
is a varied, set of item typos drawn from a relatively 
large number of cells of the matrix. The second sect. ion, 
items 41-52 on both forms, contains computational prob- 
lems in addition, subtraction, multiplication, and divi. 
si on. Thc.se questions required students to produce the 
answers rather than select them from several alternatives. 
The first two sections were administered on a pretest - 
posttest basis to all subjects. The final section, 
consisting of an additional fifteen items, contains 
additional questions selected from the matrix after 
examination of instructional materials which were devel- 
oped after the pretesting. This last section of the 
test was of course administered only on a post test basis. 
(2) A number of measures of level of cognitive 
development were administ ered to samples of exper- 
imental and comparison students at each school. 

These measures will be described only briefly in 
the preliminary report, as their analysis is pres- 
ently underway . The individually administered 
problem situations, derived mainly from the work of 
Piaget and Bruner, provide measures of generalized 
cognitive functions rather than indications of 
level of school achievement as would be inferred 
from the LAMM] 1 I)i agues t ic. lest or the I owa 1 e s t 



of Basic Skills. As such, the cognitive mcasuies 
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reflect stages in intellectual growth as repre- 
sented iji developmental theory rather than stages 
of achievement in school subject matter as reflected 
in grade norms , 

There were two purposes in administering the cognitive 
tests. First, performance on such measures will make 
it possible to describe the’ LAMMP and comparison groups 
with regard to a number of generalized cognitive skills 
presumably applicable to a variety of instructional 
content. Performance of LAMMP students will be compared 
with the approximate level of cognitive performance 
which, under typical circumstances, would be anticipated 
at the seventh grade level. Second, it may be possible 
to discover interactions between cognitive variables and 
the effects of instruction. Such information may help 
determine whether or not students with certain patterns 
of cognitive skills learn more or less than students 
characterized by other patterns. We might expect, for 
example, that students who have not developed generalized 
formal reasoning skill, as defined by tne measures listed 
below, will be poorly equipped to deal with abstract math- 
ematical content. In general, the cognitive measures have 
been administered in the hope of learning whether the 
instructional programs under study are appropriate for 
the entire target population or only for sub-groups of 
that population. 



o 




t 



1 5 



The cognitive measures can be divided into four 
types: Measures of the corrj^\>o>i^n^.or. ort^3_ai>d 

c ardi nal numbers are based on the student's ability 
to understand that only when each element of a series 
is combined with the preceding ones can its position 
be determined, and only their position differentiates 
the units, which in other respects are equivalent. 
Measures of conse rv ation indicate the extent to which 
the student's grasp of the idea of quantity is in- 
variant with respect to observable physical transforma- 
tions. The measure of formal reasonin g has to do with 
the ability to generate abstract rules or principles 
that explain observed events. Such rules are not 
simply the summary of experience, but rather represent 
a formal understanding of the principles underlying a 
set of events. l : or example, the law of specific gravity 
is a generalized explanation of how it is that any 
object will sink or float in a liquid. A probability 
statement is a rule for predicting the frequency or 
likelihood of specific events. Both of these types 
of rules are arrived at through what Piaget has termed 
formal reasoning. Finally, the measures of £l°uplM 
reflect the levels at which people organize or categorize 



objects and events 
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Solid': on and card inat ion 



;ve given bclo. 



S .-r ? at i oi: (Cards) The student is presented with 

ten cards lettered from A-J, representing units from one to ten 
respectively. The student is ashed how many units different cards 
represent, first while i n order, then again after they me di> 
arranged . 

Conservation 



Con serv ation of area: The student is asked to compare the areas 

of two fields of grass (sheets oC green papei), which are acuta! ly 
the same size. On one field houses are placed close together in 
rows. On the other fie!d the houses are spread about over the 

whole area. 

Conservation of weight : The student is asked to compare the 

weights of two equal balls of clay, one of which is flattened and 
the other of which, is left in the shape of a ball. 

Conservation of volume : (Displacement of Volume) Ihc siuocnt 

observes how far the water in a cup rises when a ball of clay is 
dropped into it. He is then asked to predict whether the water 
will rise the same amount when the ball of clay is flattened 
before it is dropped into the cup. 
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(V. .1.1 i I..-. ! o r i ;* 1 jirnbj cm : fro;.- 

d i ■ j c r e i j t c e 3 o r s t h c stud c t 
c o 1 o ;« c oiab j lit- 1 • o :i r. . 



souares of railroad boaid in six 
is aslxd to fo-tke all the possible 



1 lo: l ing bodies: The student is presented w.i th v 

and asked whether or not they will float. He is 
to pui them in a bowl of water, after which he is 
late a rule stating why things float. 



arioos objects 
then instructed 
asked to formu- 



Group i ng 

Grouping: The student is presented with tv/o different pictures 

and is asked how they are alike, A third picture is added to the 
previous two and he is again asked how all three are alike. 

Classi ficat ion: The student is given a sheet of paper which con- 

tains geometric figures of varying size and color, lie is asked 
to group them according to their attributes. 

(3) The Social Survey Instrument included items on: 
students’ school social history 

attitudes toward school, self and world outlook 
educational materials in the home 
background characteristics of self, family 
and language behavior 

educational aspirations of self and parents 

(4) The purpose of this test is to assess children’s 
and adolescents* attitudes toward various fac- 
ets of school life as well as to other 




'Jhe test 



i • # t . 1 ' ( c 1 ue \ i ■ ud cult oral uct.i \ j l j es 



lias h.vu designed sp cifical ly so that these 
attitude- can be assessed indirectly and non- 



vc i he 1 ly 



The Attitude Toward School Test consists of 40 pictures repre- 
senting s i t uut i j is v-h'-h in c familiar to children and adolescents. 
The 40 pictures arc divided into four series of 10 pictures each 
to measure att itudes toward o. variety of activities and situation. 



Spec i f i cat 1 y : 

1. Attitudes toward school in general. 

Ibis series includes pictures of activities which occur 
in school such as assembly, the library, a cafeteria 
scene, etc 

2. Attitudes toward Social Studies. 

Pictures representing situations which occur in social 
studies classes such as looking at a map, an historical 
mural, etc. 

3 . Att i tude*s toward Mel hcinst i cs - 

In this series pictures arc included which represent 
classroom situations in which mathematical activities 
are presented, eg., writing mathematical problems on 
the board. 

4. Attitudes toward General Intellectual Activities. These 
pictures depic.1 activities which lake place outside the 
school, eg,, going to a museum, a concert, an athletic 
event . 



In each picture boys and girls arc depicted engaging in the 
various activities described above The figures have purposely 
been drawn only with general outlines, the features obscured and 
speci Tie cues limited, to reduce the possibility cf the students* 
responding to irrelevant aspects of the pictures- 

The 'indents arc told that they will look at a series of 
pictures depicting girls and hoys engaging in different activities. 



scale- (3) how the poo - 



They arc ashed to i jt*"* s o: *e oe - 5 ' C: - t;i1 l‘°- '** 
plf. in the picture fee 3 and (2) how they would fee] if they were 

in the situation depleted in tno 

Prior to adi-inisUrir.fi the tost, students are instructed that 

there are no right or v. roi.fi answers , and to respond the way they 

really feel. 

In addition to the data collected by the instruments listed 
above, information has been obtained from two othe. sources. 

First, regular observations of classroom? were made by the eval- 
uation team throughout the school year. Second, the evaluation 
team has maintained frequent contacts with teachers and central 
office project personnel from the beginning of the project in 
February 1967. Through interviews and informal discussions 
information has been collected about teachers' and supervisors' 

perceptions of the program. 

2. Pertinent dates 

a) The instruction* 1 prog rati began September 5, 190/ 

b) Evaluation data were collected with the instru- 
ments described above in a pretest period from 
September 26 to November 30 and in a posttest 
period from April 23 to May 23. Collection of 
information from interviews and informal contacts 
with project personnel began in February 1967 and 
has continued to date. 
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3 . Comp'' r i s on g r o* v* ; • 

The p'j.'oauu.i e.s for selection of comparis .on classes 
were described 5. u sect ion I 1 , ]> above. Compar j son 
classes mot duiing the same class periods as experi 
mental cl a j;;-- or. . 

Characteristics ol copper 5 son c 1 a « s c s are shown in 
Tables 5 through 8. 



Table 5 



Sex of Comparison Classes 



School 






Pal 1 , 


_J967 


Spring t 


1968 








% Girls 


% Boys 


% Girls 


% Boy 


Belvedcrq 






S3 


47 


53 


47 


Edison 






49 


S3 


45 


55 


Pacoima 






55 


45 


59 


41 








Table 


6 








Ethnic 


Groups Among Comparison 


Classes 




School 


% Caucasian 


% Negro % 


Mexican - 


American % 


Other 


Belvedere 


1 




2 


94 




3 


Edison 


1 




92 


6 




1 


Pacoima 


67 




5 


25 




3 



21 



'i ah.le 7 



Gr 


ade l.c 


] s 


of Cefrpa 


ri s', n C‘Jas.-*os 








1 ;• 1 1 , 


1967 


Sj ) r ) ng 


,1968 




Be lye cl ere 


B7 


A7 


To t a 3_ 


B7 A7 


B8 


Tots 


No. of Classes 


3 


] 


4 


3 


1 


4 


No. of Students 


S5 


26 


83 


49 


24 


73 


lid is on 














No. of Classes 


6 




6 


2 6 




8 


No. of Students 


101 




101 


33 141 


— — 


174 


Pa co j 














No. of Classes 


3 


1 


4 


3 


1 


4 


No. of Students 


86 


33 


13 9 


110 


35 


14 5 




No. 


of 


students , 


all schools 








51 




A7_ 


B8 


Total 




Fall, 1967 


242 




59 




301 




Spring, 1968 


33 




300 


59 


392 
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Table 8 



School 


Mean l.Q. by Scho 
on the C a 1 i l o x r i a 
(J 957 Short 

Verbal 

Mean 


o 1 f o r C Oii:-p arisen C 1 a s s e s 
T e s l o .f M c r» ! a 1 Ma t u r i t y 
Form). Fifth Grade. 

Non-Verbal 

Mean 


Total 

Mean 


Belvedere 


91.0 


89.0 


90.2 


Edison 


93.1 


86.8 


90.1 


Pacoima 


101.5 


99.4 


100.5 





This report contour* only the 
of data from the; Iowa Test of 



pr cl j.m i n a j • y a n a. 1 y s i s 
basic Skills, the 



LAMM P JJ i a p n o s t i c To s 1: 



and 



Hie Social Survey Instru- 



ment. The final 
analysis of data 
analysis of da t a 
development and a 
final report will 



report will include further 1 
from these instruments, plus 
from the measurement of cognitive 
ttitudes toward school . The 
a] so include some assessments 



based on information obtained from discussions 



with project personnel. 

For this report only summary statistics, i.e., mean 
and correlations have been used. There is no attempt 
at tliis point in the analysis to make statistical 
inferences . 



?4 



Sum-..: 1 ry a a d .1 j i t o v i > re. c a i. i on o ' hv a Tu -ft i op . Da i 

1. Assignment of students to experimental and comparison 
groups 

The research design adopted in the original LAlmu 
proposal required the identification of a 1 a i get 
group of students within each of the three schools, 
followed by random assignment into experimental and 
comparison groups. This procedure was adopted in 
order to provide assurance that systematic differ- 
ences did not exist at the beginning of instruction 
between experimental and comparison subjects. Dif- 
ferences in achievement observed later on thus cannot 
be attributed to differences that existed prior to 



the beginning of the program. 

An example of the type of problem caused by failure 
to achieve random assignment is provided by the fol 
lowing. Assume that in one school the comparison 
subjects arc initially higher in mathematics achieve- 
ment. For a given test item further assume that 70% 
of the comparison subjects and 40% of the experimental 



subjects answered correctly at the pretest. This 
means that a gain of only 30% is possible for the 
comparison group as compared to a possible 60-o 



for the experimental group. 



This kind of situation 



makes it relatively easy for the experimental 



group to look superior with respect to gain, 
simply because initial performance was lower. 
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l a ! si'.cl) ? Ci'SC 1. 15:10 might US V.'C ] 1 lie 1 . VO bcOl S U VC‘d by 

not bet be; ing with a comparison group, since c ontrasts 
between experi ment v 1 and comparison subjects arc mnjnly 
iiiO an ing 1 o r. s o r un :i n t e rp r e t ab 1 c. . 



We re ex pc rim cn t si and comp or Ison groups in_ _ th_o 
three schoo ls a jrp r o x in i ate 3 y e quivalent at the 
begin n i n g o f the pro gram? 

Assignment of students to experimental or comparison 
classes was conducted by staff members of the school 
involved in the project. This is usually an unsatis- 
factory procedure unless explicit rules are provided 
and adhered to by the schools. Random behavior is 
simnl v not in the nature of ma.n . That this principle 
applies in the present ease is evident from the data 
in tables 9-12. 

Table 9 presents means and percentiles on the verbal 
subtest of the Iowa Test of Basic. Skills (I IBS) . While 
no differences in verbal ability at pretest are evident 
for the Belvedere students, the comparison students at 
Edison rank 6 percentile points above the experimental 
students. At Pacoima the difference is 12 percentile 
points, again in favor of the comparison subjects. 
Performance on the Arithmetic subtest of the I TBS, 
summarized in Table 10 shows the comp a r i s on subjects to 
be above the experimental subjects in all three of the 
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Table 9 



P RE T E S T P ER F 0 RMAN C i i 
SCHOOL FOR EXPER1 MENTAL AND 
SUBJECTS ON VERBAL SUBTEST OF IOWA 



BY 

COMPARISON 
OF BASIC 



TEST 



SKILLS* 



School Pretest 





Experimental 


Comparis 


Belvedere 






Mean 


17.05 


17.43 


lile rank* 


23 


23 


Edison 






’ Mean 


14.66 


16.67 


Idle rank 


17 


23 


Pacoima 






Mean 


19.83 


24.68 


lile rank 


31 


43 


Total 






Mean 


17.57 


20.0 


lile rank 


26 


31 



^Percentile ranks refer to national ITBS norms for end 
of sixth grade. 



o 

ERIC 
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Table 30 



PRli AND POST FllST PERFORMANCE BY 
SCHOOL FOR EXPERIMENTAL AND COMPARISON 
SUBJECTS ON ARITHMETIC SUBTEST OF IOV;A TEST OF BASIC SKILLS 



School 


Pretcs 


t 


Postte 


St 




Experiment a 3 


Comparison 


Experimental 


Comparison 


Belvedere 

Mean 


16.02 


16.8 


18.3 


20.44 


lile rank 


17 


23. 


26 


32 


Edison 










Mean 


14.26 


15.32 


16.04 


16.93 


%ile rank 


10 


14 


17 


21 


Pacoima 










Mean' 


16.49 


21.19 


19.27 


25.17 


lile rank 


17 


35 


30 


49 


Total 










Mean 


15.8 


18.1 


17.99 


20.89 


%ile rank 


17 


26 


26 


35 



o 

ERIC 



2 <? 



schools. The superiority of the comparison subjects is 
only 4 percentile points at Belvedere and Edison, but 
it is there. At Paco in?, the difference is very large 
indeed. 

Mean scores on the LAMMP Diagnostic Test are provided 
in Tables 11 and 12. Table 11 is based on the first 
forty items of the LAMM P Diagnostic Test while Table 12 

s 

includes the 12 additional items measuring computational 
skills. Only in the case of Belvedere does it appear 
that the two groups were at the same level of initial 
mathematics achievement. Small but consistent differ* 
ences favor comparison over experimental subjects at 
Edison. As in the case of the ITBS, the LAMMP Diagnostic 
Test at Pacoima reveals large differences in favor of 
comparison subjects.. 

Initial differences between experimental and comparison 
groups also appear in the data from the Social Survey' 
Instrument. Some differences, small but consistent, 
arc seen in all three schools. (Table 13) 

Experimental students in all three schools showed 
higher school mobility than comparison students. Educ- 
cational aspirations of experimental students themselves 
and their parents* aspirations for them were lower than 
the same aspirations for comparison students. Other 
differences show up when the schools are considered one 



Tabic 11 



PIIF AN)) POSTTHST PERFORMANCE BY SCHOOL 
for EXPERIMENTAL AND COM) ‘ARISON SUBJECTS 
ON LAMMP DIAGNOSTIC TEST (ITEMS 1-40)'- 



School 





Pretest 




Posttest 

» . ■** f* m h r* **■% 




Experimental 
Fm A Fm B 


Compar j son 
Fin A Fm B 


Experimental 
Fm A Fm B 

• 


V/Omp t.t A J 

Fm A Fm B 


Belvedere 

mean 

sigma 

N 


23.1 

X9 

60 


24,2 
5 . 6 
63 


24.7 23.4 
578 "3.5 
28 31 


25.7 

“5.8 

60 


27.6 

5.3 

63 


31.3 25.9 

3 . 8 4.7 

27 32 


Edison 

mean 

sigma 

N 


21.9 

4.1 

21 


21.2 

“4.8 

32 


24.7 21.7 
'4.4 5.6 

32 24 


25.3 

4.8 

24 


26.4 

5.3 

34 


26.5 28.9 

5.2" 3.9' 

33 18 


Pacoima 

mean 

sigma 

N 


26.8 

T79 

56 


24.8 

~S7T 

53 


30. S 29.3 
"4 . 4 O" 
41 , 47 


27.7 

5.0 

57 


27 A 

~T74 

52 


31.6 32.4 

4.5 T.6 

41 47 



*The data reported in Table 11 are based only “ 

students who were in school for the entire 1967-1968 year. 
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Tab lo 12 

PI Ui AND POSTTKST PERFORMANCE BY SCHOOL 
FOR EXPERIMENTAL AND COMPARISON SUBJECTS 
ON LAM;.iP DIAGNOSTIC TEST (ITEMS 1-52)* 



School 


Pretest 


Posttcst 




Experimental 


Comparison 


Experimental 


Comparison 




Mil A 


Fm B 


Fm A 


Fm B 


Fm A 


Fm B 


Fm A 


Fm B 


Belvedere 


















mean 


29.3 


31.6 


32.1 


31.1 


32.1 


34.9 


39.6 


33.9 


sigma 


8.1 


6.7 


6.3 


4.3 


6.7 


6 . 5 


5.3 


575 


N 


60 


63 


28 


31 


60 


63 


27 


32 


Edison 


















mean 


27.7 


27.5 


31.4 


28.5 


32.1 


34.3 


33.9 


37.4 


sigma 


4.5' 


6.1 


5.0 


“771 


*“5~5 


6.1 


“F7T 


~57*2 


N 


21 


32 


32 


24 


24 


34 


33 


18 


Pacoima 


















mean 


33.1 


32.2 


38.0 


37.6 


34. S 


35.4 


39.2 


40.6 


sigma 


5.6 


6.2 


5 .1 


“571 


5.5" 


5.3 


6.2 


5.5 


N 


56 


53 

; 


41 


47 


57 


52 


41 


47 



*Thc data reported in Table are based only on those students 
who wore in school for the entire 1967-1968 year. 
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at a time. At Edison the experimental students ex- 
pressed less liking of school, less trust of people and 
more feeling of pov;erlcssncss than comparison students 
(Table 14). (It is interesting to thinlc of how these 
characteristics might have been unconsciously taken into 
account by the teachers who assigned students to experi- 
mental and comparison classes.) At Pacoime experimental 
students had more Spanish-speaking friends than compari- 
son students (6 7%* experimental, 531 comparison). Even 
at Belvedere the experimental students expressed less 
trust of people than comparison students (501 experi- 
mental, 65% comparison), and more experimental students 
• came from homes where Spanish rather than English was 
spoken (64% experimental, 47% comparison). Like the 
dkta from the mathematics tests, the results of the 
Social Survey Instrument tend to show. the comparison 
students in a more favorable position than experimental 
students as far as an academic stiuation is concerned. 

The implication of these data is that subjects were not 
randomly assigned to experimental and comparison conditions at 
Edison and Pacoima. This renders invalid any inferences about 
the effects of instructional variables based upon differences 
between experimental and comparison subjects. 




Table 13 



PERCENTAGE 01 : STUDENTS EXPRESSING 
AGUEEMEHT KITH SELECTED ITEMS 
ON SOCIAL SURVEY INSTRUMENT , PRETEST 



5 


Bel ve 


dcrc : 


Edison j 


Pacoima 


Item 


Exp . 


Comp . 


Exp . 


Com£. 


S*E> 


Comp , 


Have attended 

3 or more schools 


56 


48 


67 


57 


55 


46 


I would like to go 
to college 


63 


75 


76 


90 


62 


74 


Parents would 
like me to go 
to college 


68 

— — — — 


78 


81 


91 


70 


80 



Table 14 

PERCENTAGE OF STUDENTS AT EDISON 
EXPRESSING AGREEMENT KITH SELECTED 
ITEMS ON SOCIAL SURVEY INSTRUMENT, PRETEST 



Item 


Experimental 


Comparison 


Like school 


63 


80 


People can be trusted 


42 


55 


There isn*t much people 
can do about the way 
things are going to turn 


77 


61 


out 
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2 . hie as urc mcnt o :C Lea r acing in Mat hema tie s 

a) Teacher ratings of relevancy of L AMMP D i a g nostic 



Test items 

Insofar as possible items comprising forms A and 
B of the LAMM P Diagnostic Test were selected to 
be representative of instructional goals of the 
LAMMP project. However, the major proportion of 
the items were selected during the summer of 1967 
before it was possible to inspect LAMMP instruc- 
tional materials, the bulk of which were developed 
later on during the 1967-68 school year. Decisions 
as to the content of the LAMMP Diagnostic Test 
were therefore guided by information gleaned during 
early planning meetings of the LAMMP teaching staff 
and from those instructional materials developed 
for the initial stages of the program. Since some 
assumptions had to be made about test content, there 
were grounds for concern about the test f s relevancy 
as a major evaluation instrument for the LAMMP 
program. 

This matter was investigated by having teachers of 
both experimental and comparison classes evaluate 
the relevance of each item of the LAMMP Diagnostic 
Test. Shortly before the end of the spring term 



the teachers were asked to judge the relevancy of 
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instruction 
Test item? 



in the j r desses to LAMM P Diagnostic 

m ~~ " ' ** ’** r~ *’’** 

during the preceding year. Specifically, 



they were instructed to, 

"Make a judgment on the extent to which 
instruction in your mathematic classes 
this year would facilitate student's 
ability to answer each item correctly • " 

The following 5-point rating scale was provided. 



Instruction in rny clashes this year 



Definitely Probably would Uncertain 

should not not 

facilitate facilitate 

ability to 

answer 



Probably 

should 

facilitate 



Definitely 
should 
facilitate 
ability to 
answer 




Thirteen teachers of experimental and 13 teachers 



of comparison classes agreed to complete the judg- 
ments. Mean ratings for each of the 52 items 
administered in the pretest were computed separately 



for experimental and comparison teachers within 
each school. These and other related data were 



analyzed for the purpose of answering three ques- 
tions . 



(1) Did the teachers view the test as relevant to 
their instruction and were there differences 
in the ratings of experimental and comparison 
teachers? 

Mean ratings averaged over the 52 items in 
each form of the LAMMP Diagnostic Pretest 



0 
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are presented in Table 15. These ratings, 
identified according to school and experimen- 
tal versus comparison teachers, show that 
the test as a whole v:as judged to be relevant 
to instructional goals as defined by the 
teachers themselves. Only one of the means 
(Pacoima, comparison subjects, form B) was 
slightly b elow the midpoint of 3.0 on the 
scale of judged relevancy. One might have 
anticipated that the experimental teachers 
would have seen the test as more relevant, 
since their expressed instructional goals pro- 
vided the basis for item selection. Ihis 

J 

was certainly not the case. The ratings 
summarized in Table 13 reveal no clear pat- 
tern of differences between experimental and 
comparison ratings. Indeed, the ratings of 
comparison teachers at Belvedere are the 
highest in the table. 

(2) To what extent did experimental and compari- 
son teachers agree on the relevancy of indi- 
vidual items of the LAMMP D i agnostic Test ? 
While overall ratings of item relevancy were 
similar for the experimental and comparison 
teachers, it may be that the two groups saw 
different items as relevant. If there were 



Tabic 15 



MEAN RATINGS OE RELEVANCY 
OE LAMMP DIAGNOSTIC TEST ITEMS* 



School 


Experimental 
Em A Em B 


Comparison 
Fin A Em B 


Belvedere 


3.71 


3.81 


4.28 


4.23 




(5) 


(5) 


(3) 


(3) 


Edison 


4.0 


3.98 


3.94 


3.82 




(4) 


(4) 


(8) 


(8) 


Pacoima 


3.61 


3.36 


3.59 


2.95 




(4) 


(4) 


(2) 


(2) 



*The numbers in parenthesis indicate the number of teachers 
contributing to each mean rating. 
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such differences on a number of items it may 
be possible to .identify subsets of items on 
which the achievement of experimental and 
comparison students might be expected to dif- 
fer. To answer tills question mean ratings by 
experimental and comparison teachers of the 
52 items were inter -correlated within each 
school for each test form. These correlations 
are reported in Table 16. Only for Edison is 
there a relatively high relationship between 
the ratings of experimental and comparison 
teachers. For the other two schools the re- 
lationships, while always positive, are quite 
low. One interpretation of this result is 
the report of members of the evaluation team 
that Edison appeared to be. the only school 
where discussions about instructional goals 
and procedures occurred between experimental 
and comparison teachers. Although the LAMMP 
Diagnostic Tes t appears to be based on appro- 
priate overall content for both experimental 
and comparison classes, evidently somewhat 
different subsets of items were seen as rele- 
vant by the two groups of teachers in two of 
the three schools. 

(3) As reflected in the ratings of Diagnostic 
Test items, to what extent were curriculum 
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CORRE L AT IONS B ET WE E N 
TEACHER'S RATINGS OP 



EXPERIMENTAL AND COMPARISON 
LAMMP DIAGNOSTIC TEST ITEMS* 







Belvedere 


Ed.i son 


Pacoima 


Form 


A 


.35 


.62 


.20 


Form 


B 


.12 


.64 


.18 



♦Correlations were computed over mean ratings on 52 test items. 



Table 17 

RELATIONSHIPS BETWEEN PROPORTION 
ANSWERING ITEMS OF LAMMP DIAGNOSTIC TEST 
CORRECTLY AT PRETEST AND TEACHER'S RATINGS 
OF RELEVANCY TO INSTRUCTION 



School 


Experimental 
Fm A Fm B 


Comparison 
Fm A Fm B 


Belvedere 


.25 


.62 


• 

4^ 


.'30 


Edison 


.44 


• 

cn 

oo 


.27 


.32 


Pacoima 


.53 


.62 


.10 


.25 



goals directed at. weaknesses in mathematical 
skills exhibited by students at the beginning 
of the LAMMP program? 

Educators would doubtless agree that instruc- 
tion ought to be aimed at gaps in achievement 
rather than at what has already been mastered 
by most students. Having noted that teachers 
see the Diagnostic Test as generally reflect- 
ing their own instructional goals, it is 
legitimate and interesting to examine rela- 
tionships between the proportion . of students 
passing each item at pretest and the teacher 
ratings of item relevancy made -at the end of 
the year. Such relationships should give an 
indication of the extent to which both experi- 
mental and comparison curricula were directed 
at needs of the students. It can be argued 
that ideally the correlations reported in 
Table 17 should be negative, indicating 
that teachers place greater emphasis on those 
skills in which students are initially weak. 

\ 

Correlations of approximately zero magnitude j 

j 

j 

would indicate a lack of any relationship 1 

between student skills at entry and the ' 

instructional program, hardly . a desirable j 

situation. Positive correlations, as 

9 i 

t 

appear in Table 17, are even less desirable. j 



— - - ■ 
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Tcacla ru 
actual ly 



in cxpe rimer tal and compari son group? 
appear to have* been directing instruc- 



tion at those- areas in v.iu eh students weic 



i n i t i a 1 1 y m o r e c a p a b 1 e l M ore e v c r , s i n c e in 
every case tie coefficient on the right side 
of Table 37 is higher than the corresponding 
value at the left, there appears to have been 
a greater tendency for experimental than for 
comparison teachers to emphasize those skills 
at which students were initially, more able. 

The results reported in Table 17 have signifi- 
cant implications not only for the instruc- 
tional progi ogi under study but for educational 
practice in general, and great care must be 
taken in their interpretation. If it is really 
true that teachers in both experimental and 
comparison programs were relatively unaware 
of the particular pattern of entry skills 
characterizing their students and actually 
placed greater emphasis on topics with which 
students were already relatively more fa- 
miliar, it would be unrealistic to anticipate 
significant improvement in performance. 

Most of the time would have been spent on 
topics which students already know. Why 
might teachers do this? One explanation 



o 



lua) lu- liiai t Isc* l ask of leaching students like 
these .in the h/V-i.-iP Program ia;:> he rendered 
e as 1 e i by d i rec t j n g i ns t rue t i on a t areas of 
achievement in which some competencies al- 
ready existed. 



The above interpretation, while consistent 
villi the data of Table 17, is certainly dif- 
ficult to accept in its unequivocal form. 

One might wonder, for example, how any learn- 
ing at all can go on in the schools if the 
curricula were confined to what students al- 
ready know? Of course, we arc not dealing 
here with data taken from middle class, subur- 
ban schools. The present students arc already 
very far behind in mathematics achievement. 

In the typical situation the deficit could 
be expected to widen in the future. Thus it 
may be that the frustrations encountered in 
teaching educationally handicapped students 
plus the need perceived by the teachers to 
provide such students with success experiences 
lead in the direction of making things easier 
by placing greater emphasis on these areas of 
content in which present capabilities of stu- 
dents are most developed. 

There is at least one rather different ex- 
planation of the relationships in Table 17. 



V’e are, after all, dealing with .ratings , with 
what teachers s a 1 0 was emphasized in instruc- 
tion. Moreover, these ratings were collected 
at the end of the year after the teachers had 
ample opportunity to become familiar with 
their students* strengths and weaknesses in 
mathematics. Perhaps the ratings of relevancy 
of items do not reflect what was done in the 
classroom at all, but simply teachers* per- 
ceptions of student capabilities. This in- 
terpretation does have the advantage of pro- 
viding a possible basis for the correlations 
being higher for the experimental teachers. 

One might guess that at the end of the year 
the teachers, now quite familiar with their 
students* capacities, would unconsciously 
"take credit*' by rating those items on which 
students would do relatively better as more 
relevant to their own instructional goals. 

The experimental teachers, acutely aware of 
the attention being paid to the achievement 
of their students, might be more unconscious- 
ly motivated to present such a picture, hence 
the higher correlations on the left side of 
Table 17. 

It would be unwise on the basis of the data 
presently available to choose between the 
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two type s of exp] an<* t i ows « As a general 
issue of considerable significance* for edu- 
cational research, the relationship between 
instructional goals and student skills should 
be investigated further. For the purpose of 
the present report, the most probable con- 
clusion is that both kinds of explanations 
are valid. Teachers from both experimental 
and comparison classes were evidently in- 
sufficiently aware of the specific patterns 
of skills their students possessed at entry 
to the ? tli grade. Gross comparisons of 
total achievement test scores with national 
norms do not provide meaningful diagnostic 
information.. Finally, the correlations are 
probably increased due to the very real mo- 
tivation on the part of all teachers to 
effect improvements in their students* 
achievement . 

Conclus ions 

1) The approach used in the development of 
the hAMMP Diag nostic Test appears to be a 
useful one in the sense that both comparison 
and experimental teachers see the content as 
in general relevant to their instructional 
goals. Moreover, there appear to be no 
overall differences in the relevancy of the 
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LAMM i* Tost items as perceived by 

comparison versus experiments] teachers. In 
this sense the test is seer, as a fair one by 
both groups. 

2) In spite of the similarity of overall 
relevancy ratings by experimental and com- 
parison teachers, there is evidence that 
some differentiation among curriculum goals 
of the two groups might he made for subsets 
of items. The nature of this differentiation 
w T ill be explored in the later report and may 
provide guidelines as to the specific differ- 
ences in the instructional outcomes of ex- 
perimental and comparison classes, if such 



exist. 

3) There is no evidence that specific 
strengths and weaknesses of the student 
populations were taken into account in the 
setting of instructional goals, both in the 
case of the LAMMP and comparison classes. 
According to at least one interpretation of 
the data, there is in fact evidence to the 
contrary. Teachers may have emphasized 
students’ strengths rather than weaknesses. 
This tendency appears to he more character- 
istic of the experimental classes. While 
the finding needs to he replicated in other 
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research, it is of high relevance to the dc- 
sit»n of all kinds of instructional programs, 
in the future development of instructional 
materials for LAMMP, account .should be taken 
of available diagnostic information as to 
the entry skills of the students, 
b) Comparisons between total scores on the LAMMP 
Diagnostic Test and on the Iowa Test of Basic 



Skills 

Comparisons between pre~ and posttest scores on 
the LAMM P Diagnos tic Tes t are presented for 
Items 1-40 in Table 11 and for Items 3-52 in 
lable 12. Tables 11 and 12 have also been pre- 
sented graphically in Figures 2-5. Probably 
the only clear-cut result pertaining to achievement 
gains apparent from these data is that all groups 
made some gains over the year on overall test 
scores. There do not appear to be consistent 
differences in achievement gains between experi- 
mental and comparison groups. Because of the lack 
of comparability of experimental and comparison 
groups in two of the schools, such comparisons are 
not particularly meaningful in any case. Similar 
results arc apparent in the case of the Arithmetic 
section of the ITBS Test, as reported in Table 10. 
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Despite current practice, it is our view that 
comparisons based on total test scoics me not 
especially useful, except in a very general 
suiwnat ive sense. The content of the UW Ei. a .8' 
nostic. Test, for example, is deliberately broad 
in scope so as to reflect as many as possible of 
the goals of the three J.AMMP curriculum development 
teams. It is to be expected that gains in certain 
areas of concentration may be mainly obscured in 
a total test score by lack of gains in areas which 
received little or no emphasis. 

Because total scores arc not especially informative, 
we arc in the process of combining Dj^novtic 

Test items from the same content and/or process 



dimension of the content-process matrix in a 



n 



attempt to produce relatively homogeneous sub- 
scores, which will be more likely to reflect 
particular emphases in instruction. In this 
regard for example, we have some preliminary 
evidence of increased skills in algebraic concepts 
and operations for experimental classes in one of 
the schools and for increased achievement in con- 
cepts relating to set theory for comparison classes 
in another school. The original intention has 
been to use the teachers’ ratings of item relevance 
to identify subsets of items on which gains might 
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be expected. The useful ness of this approach is 
somewhat in doubt since, as indicated earlici , 
teachers tended to sec as relevant those items on 
which students were already performing at a rela- 
tively higher level. 

3 . M c a s u r on ) e n t o f _s tu de n t s * a ( l:i tud es 

A group of items on the Social Survey Instrument 
gives some information about students’ view of the 
world and of themselves. Pre- and posttest scores 
on these items show certain trends in all students, 
both experimental and comparison. All groups at the 
end of the year express less liking of school, less 
satisfaction with themselves, are less in agreement 
with the statement that ’’children should obey all the 
rules their parents make for them”, and are more in 
agreement with the statement, ’’The best way to set- 
tle some arguments is by a good fight”. There may 
be some clues hero about what tends to happen to 

these students in junior high school. 

There do not appear to be any strong patterns of 
differences between pre-post test scores for experi- 
mental groups as contrasted with scores for comparison 
groups. In the case of Ldison, however, there is some 
indication of a change in the outlook of experimental 
students. At the beginning of the year these students 
expressed less trust in people and more feeling of 
powerlessness than the comparison students. At the 
end of the year, agreement with the statement, 
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"In general people can be trusted", had decreased for 
both groups, but much less for the experimental group 
(471 to 37$ for experimental; 52% to 20$ for compar- 
ison). Pretest and posttest scores on the item, 

"There isn*t much people can do about the way things 
are going to turn out in life", showed a decrease in 

agreement for the experimental group (75% to 67%) and 

* 

an increase in agreement for the comparison group 
(57% to 66%). Finally, agreement with the statement, 
"There are times when I think that 1 am no good at 
all", dropped from 69$ to 41$ for the experimental 
students, while it reamined at the original level 
for comparison students (64% to 60$) . Although 
slight, the evidence does suggest that the experimental 
group at Edison may be shifting toward a more favor- 
able self-concept. 

4. Combining and contrasting data frofa different schools 

% 

Analysis based upon combined data from different 
schools are probably acceptable where the purpose is 
to provide gross summary information. In contrast, 
where the purpose is to make inferences about the 
effects of instructional conditions (e.g., all LAMMP 
students versus all comparison students), such combin- 
ations are suspect from several points of view. 

For one thing, the nature of the ultimate reference 
population is undefined. Student bodies in the three 
schools differ shaiply in ethnic character. While 

O' 



all three schools are below average in verbal and 
mathematical achievement on the Iowa Test of Basic 
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Skills , data reported in Tables 9 and 10 for Verbal 

and Arithmetic subtests do reveal differences among 

the' three institutions. With respect to the ITBS 

Verbal score, for example, experimental students 

score at the 17th, 23rd and 31st percentiles for 

* 

Edison, Belvedere, and Pacoima, respectively. In 
the case of the ITBS Arithmetic subtest the experi- 
mental students at the 3 schools fell at the 17th 
percentile for Belvedere and Pacoima, but at the 10th 
percentile for Edison. Differences were also apparent 
for number of items correct at pretest on the LAMMP 
Diagnostic Test (Tables 11, 12). 

Similar conclusions vrould doubtless be reached from 
an examination of the cultural and demographic vari- 
ables. For example, on the Social Survey Instrument 
of those students aware of their father's educational 
history, 481 at Belvedere, 261 at Edison, and 36t at 
Pacoima reported that their fathers had less than a 
high school education. (The relative ranking of the 
schools might change if all the students knew the 
number of years of education received by their 
fathers) . 

•Since the student bodies of the three schools differ 
in so many ways, it is likewise not appropriate to 



make distinctions between schools as to the. relative 
effectiveness of LAMMP instructional programs. For 
example, measures of student achievement cannot 
provide a basis for concluding that the LAMMP program 
at Belvedere, emphasizing a "mathematics laboratory 
approach", is superior or inferior to the media- 
based program at Pacoima. If the differential 
effectiveness of the three LAMMP instructional pro- 
grams had been of interest, a quite different and far 
more complex experimental design should have been 
adopted. In any case, comparative questions of this 
sort are usually of minor importance and rarely jus- 
tify the trouble that must be taken in answering 
them. The most generalized objective of the LAMMP 
program presumably is to produce effective in- 
structional materials in all three of the Centers. 

If it is at some point necessary to choose one 
approach over the others, comparative levels of 
student achievement will probably not turn out to 
be a decisive factor. Such a decision is more likely 
to be based on considerations like cost of materials 
and equipment, training requirements for staff, and 



similar matters. 



V . Recommen d a 1 5 ons 

The following points should be taken into account, either 
with respect to the future operation. of the LAMM I? program 1 or 
with reference to the evaluation procedures by which its out- 
comes are to be monitored. 

A. In order to insure random assignment .of students to 
experimental and comparison groups , it is recommended 
that assignment to- classes for. the 1968 r 69 school year 
be done under the 'supervision of appropriate technical 
personnel from the Los Angeles City Schools or from 
the LAMMP evaluation team. 

* B. It is of course hoped that the evaluation report for 

the first year will provide some useful suggestions for 
future development of the program, but. in. the main, the 
role of the evaluation' team during this first year was 
perceived as that of providing evaluative data with 
respect to relatively long term outcomes .rather than 
facilitating the day to day processes by which desirable 
outcomes are achieved. 

This approach is described as "stimulative'! in the litera- 
ture of evaluation. In its more negative aspects this 
sort of evaluation can be likened to. the .role of St. 
Peter at the Golden Gates, judging the .product but not 
participating in its shaping. A very different approach 
has been termed "formative" evaluation. . In its extreme 
form, formative evaluation is concerned solely with 
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developing the best possible product, in the hope that 

the product will meet whatever summative standards are 

later imposed. In this sense the formative evaluator 

# 

is more like the parish priest who does his best to 
guide the member of his congregation along the path of 
righteousness, but who, whatever the quality of the 
finished product, acts as the advocate at the critical 
moment . 

Some of the experiences gained .during the initial year 
of the LAMMP project have led members of the evaluation 
team to feel it would be desirable. in the future to _ 
engage, at least to some extent, in a more formative 
role, vis a vis the LAMMP program. Although we would 
see the evaluation effort next year as having both 
summative and formative aspects, we do sense that the 
interests of LAMMP and the Center for the Study of 
Evaluation of Instructional Programs would be benefited 
by a closer connection between the evaluation and the 
process of developing the LAMMP instructional program. 
This need is especially apparent in the. seeming lack of 
mesh between student skills at entity and specific in- 
structional goals, as indicated by the teachers* 
ratings of LAMMP Diagnostic Test items*. Had diagnostic 
information been communicated to the teachers in the 
initial phases of the program, there might have been an 
earlier re-assessment of instructional objectives. 
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We therefore propose that for the next year a procedure 
be worked out for providing teachers with feedback as 
to student accomplishment. One way of doing this would 
be to have the teachers themselves examine the revised 
content -process matrix, which was used to construct the 
LAMMP Diagnostic Test , and select those areas in which 
they hope to effect improvements in student achievement. 
This process should be guided by what we have learned 
this year about the entry skills of LAMMP .students . The 
evaluation team would then develop appropriate tests for 
administration at two or three points during the year, 
in an effort /to help LAMMP teachers monitor the progress 
of their students in the 'light of their own instructional 
objectives. This kind of activity on the .part of the 
evaluation team need not interfere with the requirements 
of the more summative type of evaluation. 

An even more direct approach to formative evaluation 
would involve working with LAMMP teachers in developing 
tests for assessing student progress with respect to 
units of instruction extending over relatively brief 
periods of time. 

There are some strong arguments in favor of this approach. 
First, the more frequent testing would increase the 
teachers’ opportunities to discover and provide for in- 
dividual needs. The workshop sessions with Dr. Benjamin 
Bloom last summer represented an attempt to orient 
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teachers to this kind of procedure. Second, . in order 
to assess the effectiveness of teacher -.made materials 
at successive stages in their development, it will be 
necessary for teachers to gather systematic evaluative 
data as they try out and modify the materials. Since 
it is impractical for the evaluation staff to try to 
prepare all the tests which would be needed, we suggest 
that the evaluation staff serve in a consultant capacity 
to the teachers in a joint effort at formative evalua*' 
tion. 

C. Although this report has not presented systematically 
the data available from informal observations of the 
course of this project, two recommendations seem ob- 
vious from even a cursory consideration of the context 
within which the instructional program is proceeding. 

1, It is recommended that the time of the resource 
personnel (e.g., counselor, mathematics consultant, 
illustrators) be distributed so. that most of it is 
devoted to actual work with teachers in the three 
centers. It is also recommended that. any vacancy 
which occurs among resource personnel be filled 
rapidly so that teachers will not be left for long 
periods without the assistance they need in order 
to accomplish the objectives of the program. 

2. Conditions necessary for attaining the dual objec- 
tives of instruction and the creation of new materi- 
als by the project teachers should probably be 




59 



re-examined. The notion of teachers .creating and 
trying out new instructional materials in close 
conjunction with teaching classes is an interesting 
idea to explore. To be successful, however, it 
may require a more flexible organization than that 
used this year. A systematic effort may have to 
be made to discover various balances between 
teaching and working on materials which would be 
favorable to the accomplishment of both objectives. 
For example, teachers in each center might rotate 
assignments to periods of concentration on de- 
veloping materials and periods of testing materials 
in teaching; teaching assistants might be employed 
in a variety of ways; or different patterns of 
organization might be tried in different centers. 

In any event, consideration should be given to the 
kinds of difficulties which were apparent this 
year in teachers’ efforts to accomplish both ob- 
jectives. First, as noted above,. the special 
resources which have been built into the project 
to help in the development of materials need to 
be available to teachers on a more frequent and 
reliable basis. Second, the cost in teacher time 
which is required by publicity efforts, especially 
large numbers of visitors, and by frequent in- 
service training activities should be weighed 




against the amount of teacher time required to accomplish 
both objectives of the program. 

If the formative evaluation approach, which has been 
recommended, is adopted,' it will also .be important to 
provide teachers with enough time to work with the 
evaluation team and on their own in the preparation of 
testing materials. This additional demand on time 
could bring large benefits in the better monitoring of 
pupil progress and the adequate assessment of newly 
created instructional materials. 



